Electronic device and method for image processing

ABSTRACT

A method for displaying images is provided, including displaying, by an electronic device, a first image obtained from an image sensor; displaying, by the electronic device, a plurality of second images together with the first image, wherein each second image is generated based on the first image and a respective image filter; and responsive to a selection of at least one second image, generating a third image based on the first image and the selected second image&#39;s respective image filter and displaying the third image in place of the first image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.16/402,503 filed on May 3, 2019 which claims the benefit of the earlierU.S. patent application Ser. No. 15/588,882 filed on May 8, 2017 andassigned U.S. Pat. No. 10,284,788 issued on May 7, 2019 which is aContinuation of U.S. patent application Ser. No. 15/364,423 filed onNov. 30, 2016 and assigned U.S. Pat. No. 9,674,462 issued on Jun. 6,2017 which is a Continuation of U.S. patent application Ser. No.14/208,719 filed on Mar. 13, 2014 and assigned U.S. Pat. No. 9,571,736issued on Feb. 14, 2017 which claims the benefit under 35 U.S.C. §119(a) of a Korean Patent Application filed in the Korean IntellectualProperty Office on Mar. 14, 2013 and assigned Ser. No. 10-2013-0027590,the contents of which are herein incorporated by reference.

BACKGROUND 1. Field of the Disclosure

The present disclosure relates to electronic devices, and moreparticularly to an electronic device having an image processingfunction.

2. Description of the Related Art

Electronic devices such as cellular phones, electronic schedulers,personal complex terminals, laptop computers and the like have become anecessity to modern life. Such 25 devices often include a camerafunction that permits users to capture images and enhance the capturedimages by applying various photographic effects to them. At present, theneed exists for new user interfaces for applying photographic effects toimages that are both effective and easy-to-use.

SUMMARY

The present disclosure addresses this need. In accordance with oneaspect of the disclosure, a method for displaying images is provided,the method comprising: displaying, by an electronic device, a firstimage obtained from an image sensor; displaying, by the electronicdevice, a plurality of second images together with the first image,wherein each second image is generated based on the first image and arespective image filter; and responsive to a selection of at least onesecond image, generating a third image based on the first image and theselected second image's respective image filter and displaying the thirdimage in place of the first image.

In accordance with another aspect of the disclosure, an electronicdevice is provided comprising: a display module; an image sensor foracquiring a first image; and processing circuitry configured to:display, on the display unit, a first image obtained from the imagesensor; display, on the display unit, a plurality of second imagestogether with the first image, wherein each second image is generatedbased on the first image and a respective image filter; and responsiveto a selection of at least one second image, generate a third imagebased on the first image and the selected second image's respectiveimage filter and display the third image in place of the first image.

In accordance with yet another aspect of the disclosure, an electronicdevice is provided comprising: an image sensor for acquiring an image;one or more processors; a memory for storing processor-executableinstructions, which when executed by the one or more processors causethe electronic device to: display, on the display unit, a first imageobtained from an image sensor; display, on the display unit, a pluralityof second images together with the first image, wherein each secondimage is generated based on the first image and a respective imagefilter; and responsive to a selection of at least one second image,generate a third image based on the first image and the selected secondimage's respective image filter and display the third image in place ofthe first image.

In accordance with yet another aspect of the disclosure, an electronicdevice is provided comprising: a processor unit configured to obtain afirst image through an image sensor, display at least one second imagecorresponding to effect for the first image, together with the firstimage, and, in response to a selection of the at least one second image,display at least a portion of the first image using the effect; and adisplay module configured to display the at least one second imagecorresponding to effect for the first image and the first image.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentdisclosure will become more apparent from the following detaileddescription when taken in conjunction with the accompanying drawings inwhich:

FIG. 1 is a diagram of an electronic device according to aspects of thedisclosure;

FIG. 2 is a block diagram of a filter management module according toaspects of the disclosure;

FIG. 3 is a block diagram of an image processing module according toaspects of the disclosure;

FIG. 4 is a flowchart of a process for image processing according toaspects of the disclosure;

FIG. 5, FIG. 6 and FIG. 7 are flowcharts of processes for displaying asecond image according to aspects of the disclosure;

FIG. 8 and FIG. 9 are flowcharts of processes for displaying imagesaccording to aspects of the disclosure;

FIG. 10 and FIG. 11 are flowcharts of processes for arranging secondimages according to aspects of the disclosure;

FIG. 12 is a flowchart of a process for displaying second imagesaccording to aspects of the disclosure; and

FIG. 13, FIG. 14, FIG. 15, FIG. 16, FIG. 17, FIG. 18, FIG. 19, FIG. 20and FIG. 21 are diagrams of different screens of a camera applicationaccording to aspects of the disclosure.

DETAILED DESCRIPTION

Aspects of the present disclosure will be described herein below withreference to the accompanying drawings. In the following description,well-known functions or constructions are not described in detail sincethey would obscure the disclosure in unnecessary detail. And, termsdescribed below, which are defined considering functions in the presentdisclosure, may be different depending on user and operator's intentionor practice. Therefore, the terms should be defined on the basis of thedisclosure throughout this specification.

FIG. 1 is a diagram of an electronic device 100 according to aspects ofthe disclosure. The electronic device 100 may include a mobile phone, amobile pad, a media player, a tablet computer, a handheld computer, or aPersonal Digital Assistant (PDA), and or any other suitable electronicdevice. The electronic device 100 may include a host device 110, anexternal memory device 121, an image sensor 122, a sensor device 123, aGlobal Positioning System (GPS) reception device 124, a wirelesscommunication device 125, an audio device 126, an external port device127, a touch screen device 128, other input/control devices 129, and apower supply device 130.

The host device 110 may include an internal memory 111, one or moreprocessors 112, and an interface 113. The internal memory 111, the oneor more processors 112, and the interface 113 may be separateconstituent elements or they may be integrated together in one or moreintegrated circuits.

The processor 112 may include any suitable type of processing circuitry,such as an ARM-based processor, a MIPS-based processor, an x86processor, an FPGA, or another electronic circuit. In someimplementations, the processor 112 may be part of a system-on-a-chip(SoC). In operation, the processor 112 may execute various softwareprograms and perform various functions for the electronic device 100,and perform processing and control for voice communication, videocommunication, and data communication. Also, in addition to this generalfunction, the processor 112 may execute a software program (i.e., aninstruction set) stored in the internal memory 111 and/or externalmemory device 121. In addition, the processor 112 may play even a roleof executing a specific software program (i.e., an instruction set)stored in the internal memory 111 and/or external memory device 121 andperform specific various functions corresponding to the executedprogram. For example, the processor 112 may interwork with the softwareprograms stored in the internal memory 111 and/or external memory device121 and perform a method of an exemplary aspect. Also, the processor 112may include one or more Application Processor Units (APUs), GraphicsProcessor Units (GPUs), audio processor units, communication processor,and/or any suitable type of electronic circuit.

The application processor unit may execute an Operating System (OS) andperform a variety of functions, and it may be one chip gathering all offunctions of a core, a memory, a display system/controller, a multimediaencoding/decoding codec, a 2-Dimensional/3-Diminsional (2D/3D)accelerator engine, an Image Signal Processor (ISP), a camera, an audio,a modem, various high and low speed serial/parallel connectivityinterfaces and the like. For example, the application processor unit maydrive the OS and applications, and may be a System-On-A-Chip (SOC) beingone chip gathering various functions of controlling a systemdevice/interface.

The graphics processor unit may perform operations related to graphicsprocessing, and may take charge of image information processing,acceleration, signal conversion, screen output and the like. In someimplementations, the graphics processor may be able to process 2D or 3Dgraphics faster than the application processor unit.

The audio processor unit may perform operations related to audioprocessing, and may change an audio signal from a digital format toanalog format (and vice versa). The communication processor unit maytake charge of communication processing between the electronic device100 and other devices and, when the electronic device 100 intends to usea communication function of a network, the communication processor unitmay perform a function of establishing a connection between theelectronic device 100 and the like. The interface 113 may connect thehost device 110 with various devices of the electronic device 100.

The external memory device 121 may include any suitable type of volatileand non-volatile memory. For example, the external memory device 121 mayinclude high-speed random access memories and/or non-volatile memoriessuch as one or more magnetic disc storage devices, one or more opticalstorage devices and/or flash memories (e.g., NAND, NOR). The externalmemory device 121 stores software. The software may include an OSprogram, a communication program, a camera program, a graphic program,one or more application programs, a user interface program, a codecprogram and the like. The term of program may be expressed as a set ofinstructions, or an instruction set, or a program. The communicationprogram, the camera program, the graphic program, the one or moreapplication programs, the user interface program, and the codec programmay use various functions of the OS program through various ApplicationProgramming Interfaces (APIs).

The OS program represents an embedded operating system such as WINDOWS,LINUX, Darwin, RTXC, UNIX, OS X, or VxWorks, and may include varioussoftware constituent elements controlling general system operation.Control of the general system operation may include memory managementand control, storage hardware (device) control and management, powercontrol and management and the like. Also, the OS program may facilitatecommunications between various hardware (devices) and softwareconstituent elements (modules).

The communication program may enable a communication with a remotecomputer, server, or electronic device through the wirelesscommunication device 125 or the external port device 127.

The camera program may include a camera-related software constituentelement for enabling camera related processes and functions. The cameraprogram may perform pre-processing for applying various effects to animage obtained from the image sensor 122 under API support of OpenGraphics Library (OpenGL), DirectX and the like, and performpost-processing for applying various effects to a captured snap image.Filters for those effects may be collectively managed such that thefilters may be commonly used in both the pre-processing and thepost-processing. In addition, the filters may be used even by otherprograms in addition to the camera program.

The graphic program may include various software constituent elementsfor providing and displaying a graphic on the touch screen device 128.The graphic program may include various APIs, such as OpenGL, DirectXand the like, to create graphics. Additionally or alternatively, thegraphic program may provide various filters which may apply variouseffects to an image. It should be understood, that the term “graphics”refers to any type of information that is renderable on a displayscreen, and it may include a text, a web page, an icon, a digital image,a video, an animation and the like. In some implementations, the graphicprogram may be an image viewer that focuses usability to imagepost-processing, an image edition program, a camera related program thatis optimized to image pre-processing, a video phone related program andthe like. The graphic program may perform post-processing for applyingvarious effects to a rendering-completed image, or may performpre-processing for applying various effects to an image. As describedabove, the filters for those effects may be collectively managed suchthat the filters may be used commonly with other programs.

The camera program and/or the graphic program may perform a function ofdownloading and installing a filter package file from the external(e.g., a network, an e-mail, a messenger, a detachable external memoryor the like). If the camera program and/or the graphic program downloadsand installs the filter package file from the network, a filter andfilter-related data (e.g., an icon image, a filter name, a filterversion and the like) which are included in the filter package file maybe managed collectively in a database. Also, when the filter is deletedfrom the database, not only the filter but the filter package file maybe all deleted. The camera program and/or the graphic program maydisplay the icon image and the filter name which indicate the addedfilter, together. The filter name may be translated and displayed in auser language. Particularly, the camera program may apply the effect ofthe added filter to an image obtained from the image sensor 122 andthen, provide the image by an icon (e.g., a thumbnail image).

The application program may include a browser, an e-mail, an instantmessage, word processing, keyboard emulation, an address book, a touchlist, a widget, Digital Right Management (DRM), voice recognition, voicereplication, a position determining function, a location-based service,and or any other suitable software.

The user interface program may include various software constituentelements related to a user interface. Also, the user interface programmay include information specifying the manner in which state changes inthe user interface are to take place. Such information may specifyconditions that trigger the state changes, and or any other suitableinformation.

The codec program may include a software constituent element related toencoding and decoding of a video file.

The external memory device 121 may further include programs(instructions) in addition to the aforementioned programs. Also, variousfunctions of the electronic device 100 may be implemented in hardwareincluding one or more stream processing, Application Specific IntegratedCircuits (ASICs), software, and/or a combination of software andhardware.

The image sensor 122 may convert light reflected from a shot object intoan electrical signal. The image sensor 122 may be a Charged CoupledDevice (CCD), a Complementary Metal-Oxide Semiconductor (CMOS) or thelike.

Various constituent elements of the electronic device 100 may beconnected with one another through one or more communication buses (notdenoted by reference numerals) or stream lines (not denoted by referencenumerals).

The sensor device 123 may include a motion sensor, an optical sensor, atemperature sensor and the like and enable various functions. Forexample, the motion sensor may sense a motion of the electronic device100, and the optical sensor may sense ambient light.

The GPS reception device 124 may receive signals sent from GPSsatellites and determine a current position.

The wireless communication device 125 may enable wireless communication,and may include a wireless frequency transmitter/receiver and an optical(e.g., infrared) transmitter/receiver. The wireless communication device125 may include a module that includes at least one antenna, a RadioFrequency (RF) connector and the like. The wireless communication device125 may be designed to operate through one of a Global System for MobileCommunication (GSM) network, an Enhanced Data GSM Environment (EDGE)network, a Code Division Multiple Access (CDMA) network, a Wireless-CodeDivision Multiple Access (W-CDMA) network, a Long Term Evolution (LTE)network, an Orthogonal Frequency Division Multiple Access (OFDMA)network, a Wireless Fidelity (Wi-Fi) network, a Wirelessinteroperability for Microwave Access (WiMAX) network, a Bluetoothnetwork or/and the like according to a communication network.

The audio device 126 is connected to a speaker 1261 and a microphone1262 to take charge of audio input and output of voice recognition,voice replication, digital recording, call function and the like, andmay include an audio amplifier. The audio device 126 may communicatewith a user through the speaker 1261 and the microphone 1262. Also, theaudio device 126 may receive a data signal from the host device 110,convert the received data signal into an electrical signal, and outputthe converted electrical signal through the speaker 1261. The speaker1261 may convert an electrical signal into an audible frequency band andoutput the audible frequency band, and the microphone 1262 may convert asound wave forwarded from a character or other sound sources into anelectrical signal. Also, the audio device 126 may receive the electricalsignal from the microphone 1262, convert the received electrical signalinto an audio data signal, and transmit the converted audio data signalto the host device 110. The audio device 126 may include an earphone,headphone, or headset that is detachable from the electronic device 100.

The external port device 127 may direct connect the electronic device100 with other electronic devices, or indirect connect the electronicdevice 100 with other electronic devices through a network (for example,the Internet, an intranet, a wireless LAN and the like).

The touch screen device 128 may provide an input and output interfacebetween the user device 100 and a user. The touch screen device 128 mayapply a touch-sensitive technology, and may forward a user's touch inputto the host device 110 and show to the user visual information providedfrom the host device 110, for example, a text, a graphic, a video andthe like. In some implementations, the touch screen device 128 mayinclude a display displaying an image. The display may be one of anElectro Wetting Display (EWD), E-Paper, a Plasma Display Panel (PDP), aLiquid Crystal Display (LCD), an Organic Light Emitting Diode (OLED),and an Active Matrix Organic Light Emitting Diode (AMOLED), and/or anyother suitable type of display. In addition, the touch screen device mayinclude a touch sensitive device, such as a stylus-pen digitizer, acapacitive overlay touch panel, a resistance overlay touch panel, asurface acoustic wave touch panel, and an infrared beam touch panel.Although in this example, a touch screen device is used that includesboth display and touch sensing components, in other examples onlydisplay device may be used. In such examples, a mouse or another type ofinput device may be used to receive user input.

The other input/control devices 129 may include an up/down button forvolume control. In addition to this, the other input/control devices 129may include at least one of a push button of a specific function, alocker button, a rocker switch, a thumb-wheel, a dial, a stick, and apointer device such as a stylus and the like.

The power supply device 130 may include at least one Power ManagementIntegrated Circuit (PMIC). The PMIC may adjust a battery power. Forexample, according to a load to process, the processor 112 may transmitan interface signal dependent on the load to the power supply device130. The power supply device 130 may adjust a core voltage supplied tothe processor 112, adaptive to the processor 112. The processor 112 maybe driven at a minimum power all the time. In addition to the PMIC forthe host device 110, at least one PMIC related to at least one of theexternal memory device 121, the image sensor 122, the sensor device 123,the GPS reception device 124, the wireless communication device 125, theaudio device 126, the external port device 127, the touch screen device128, and the other input/control devices 129 may be constructed. Also,one integrated PMIC may be constructed. The integrated PMIC may adjust abattery power of not only the host device 110 but also at least one ofthe external memory device 121, the image sensor 122, the sensor device123, the GPS reception device 124, the wireless communication device125, the audio device 126, the external port device 127, the touchscreen device 128, and the other input/control devices 129.

FIG. 2 is a block diagram of a filter management module according toaspects of the disclosure. The filter management module may beimplemented in hardware (e.g., as one or more electronic circuits), insoftware (e.g., as processor-executable instructions that are executedby processor 120), or as a combination of software and hardware. Asillustrated, the filter management module may include an applicationunit 220, an integrated filter management unit 210, a filter directory230, and a filter download unit 240. Each of the units 220 may beimplemented in hardware, as one or more instructions that are executableby the processor 120 or as a combination of hardware and software. Theapplication unit 220 may include at least one of an applicationsupporting image pre-processing, an application supporting imagepost-processing, and an application supporting both the imagepre-processing and the image post-processing. The application unit 220may include an image viewer, an image edition application, a cameraapplication, a video phone application and the like.

The integrated filter management unit 210 may provide an image filterAPI. If there is a request for a specific effect from the applicationunit 220, the integrated filter management unit 210 may load a specificfilter from the filter directory 230, and call an API function for theloaded filter. The integrated filter management unit 210 may providevideo phone applications and other suitable software with an API forimage pre-processing or post-processing. The integrated filtermanagement unit 210 may be configured to designate a subject (i.e., aCPU or GPU) that is configured to perform the filtering operation inaccordance with the loaded filter in order to generate the effectrequested by the application unit 210.

The integrated filter management unit 210 may provide a filter editionfunction. Through the filter edition function, a user may set a favoriteeffect list, and the integrated filter management unit 210 may store thefavorite effect list in the filter directory 230. Also, the user mayselect a filter intended to be deleted through the filter editionfunction, and the integrated filter management unit 210 may delete aspecific filter from the filter directory 230. Particularly, when thefilter intended to be deleted is a filter installed through a filterpackage file (i.e., a filter installation file), the integrated filtermanagement unit 210 may delete the filter package file and guarantee astorage space. The filter package file may include a filter, a filtername, an icon image, a vendor, a filter type(pre-processing/post-processing), a version or the like. Also, the usermay select synthesizable multiple effects through the filter editionfunction, and the integrated filter management unit 210 may groupfilters for the synthesizable multiple effects, and store the groupedfilters in the filter directory 230. Also, the integrated filtermanagement unit 210 may use a specific filter commonly used for theimage pre-processing and the image post-processing, because theintegrated filter management unit 210 collectively stores and managesall of image processing related filters which work in the electronicdevice 100, in the filter directory 230.

The filter directory 230 may store a plurality of filters, and may bemanaged by the integrated filter management unit 210. The filterdownload unit 240 may provide a download function. Through the downloadfunction, a user may download a filter from the external (e.g., anetwork, an e-mail, a messenger, a detachable external memory or thelike), and the integrated filter management unit 210 may store theacquired filter in the filter directory 230. Also, the filter downloadunit 240 may further provide a filter installation function. The usermay download a filter package file from the network, and install thefilter package file through the filter installation function. If thefilter package file is installed, the integrated filter management unit210 may store a filter and filter-related data (e.g., a filter name, anicon image, a vendor, a filter type or the like) which are included inthe filter package file, in the filter directory 230.

FIG. 3 is a diagram of an image processing module according to aspectsof the disclosure. The image processing module 250 may be implemented inhardware (e.g., as one or more electronic circuits), in software (e.g.,as processor-executable instructions that are executed by processor120), or as a combination of software and hardware. As illustrated, theimage processing module 250 may include an acquisition module 260, adisplay module 270, and an effect applying module 280.

In operation, the acquisition module 260 may obtain a first imagethrough the image sensor 122. The display module 270 may display atleast one second image corresponding to effect for the first image,together with the first image. In response to a selection of the atleast one second image, the effect applying module 280 may display atleast a portion of the first image by using the effect.

FIG. 4 is a flowchart of a process for image processing according toaspects of the disclosure. In operation 401, a first image is obtainedfrom the image sensor 122. Next, in operation 403, at least one secondimage corresponding to an effect (or filter) for the first image isdisplayed together with the first image. The first image may berefreshed in real-time as signals from the image sensor 122 arereceived. The second image may be created by applying a specific effectto the first image. The second image may be displayed as an icon (e.g.,a thumbnail image). In some implementations, the second image may alsobe refreshed in real-time as signals from the image sensor 122 arereceived. In operation 405, in response to a selection of at least onesecond image, at least a portion of the first image may be redisplayedby using effect corresponding to the selected second image. In someimplementations, a third image is generated by using the effectcorresponding to the selected second image. For example, a user mayselect the second image displayed on the screen through a gesture (e.g.,a touch, hovering, a single touch, a double tap, a drag and drop,flicking or the like). If there is a selection of the second image, thethird image is generated and displayed in place of the first image. If asingle second image is selected many times, the third image is generatedby applying the effect of the second image as many times as the secondimage is selected. Additionally or alternatively, in someimplementations, a plurality of second images may be selected at once.In such instances, the third image may be generated by applying theeffect of each one of the selected second images. Thus, the selection ofmultiple second images, in this example, may cause multiple filters tobe applied to the first image when the first image is displayed. In someimplementations, the third image may be generated by applying effect(s)or filters, as discussed above, to the first image. Additionally oralternatively, in some implementations, the third image may begenerated, by applying effect(s) or filters, as discussed above, to animage frame that is captured by the image sensor 122 after the firstimage.

FIG. 5 is a flowchart of a process for displaying a second imageaccording to aspects of the disclosure. In operation 501, at least oneeffect (or filter corresponding to the effect) is loaded. Next, inoperation 503, at least one second image is generated by applying theloaded effect (or filter corresponding to the effect) to a first imagethat is acquired from the image sensor 122. Afterwards, the generatedsecond image is displayed.

FIG. 6 is a flowchart of another process for displaying a second imageaccording to aspects of the disclosure. In operation 601, an effectfilter may be acquired from an external (e.g., a network, an e-mail, amessenger, a detachable external memory or the like). After it isacquired, the effect filter may be stored in a directory thatapplications commonly use. Next, in operation 603, a second image isgenerated by applying effect of the acquired effect filter to a firstimage that is acquired from the image sensor 122. Afterwards, thegenerated second image may be displayed.

FIG. 7 is a flowchart of yet another process for displaying a secondimage according to aspects of the disclosure. In operation 701, a groupof effects (or filters corresponding to the effects) is selected (oridentified). In some implementations, the group may include multipleinstances of the same effect (or filter). Additionally or alternatively,in some implementations, the group may include different effects (orfilters). Additionally or alternatively, in some implementations, thegrouping of the effects (or filters) may be activated by the receipt ofuser input (e.g., the pressing of a camera button) after selection of aplurality of second images that were generated using the effects thatare desired to be included in the group. Next, in operation 703, a firstimage obtained from the image sensor 122 may be processed by applyingall of the effects in the group to the first image, and the resultingfiltered image may be displayed.

FIG. 8 is a flowchart of a process for displaying images according toaspects of the disclosure. In operation 801, a determination is made ifthe same second image is selected multiple times. If the same secondimage is selected multiple times, the processor 112 performs operation803. Otherwise, if the same image is not selected multiple times, theprocessor 112 may perform operation 807. Next, in operation 803, anindication of the number of times the same second image is selected isoutput for presentation to a user. After that, in operation 805, theprocessor 112 may generate a filtered image by applying, to a firstimage, the effect (or filter) corresponding to the second image as manytimes as the second image is selected. Afterwards, the generatedfiltered image may be displayed. In operation 807, the processor 112 maygenerate and subsequently display a filtered image by applying once, tothe first image, the effect (or filter) of the second image.

FIG. 9 is a flowchart of another process for displaying images accordingto aspects of the disclosure. In operation 901, a determination is madewhether there is a selection of multiple different second images. Ifdifferent second images are selected, operation 903 is performed. Inoperation 903, the effects (or filters) corresponding to each of theselected second images are applied to a first image to generate afiltered image and then the generated filtered image is displayed.Otherwise, if multiple images are not selected, the process discussedwith respect to FIG. 8 is performed.

FIG. 10 is a flowchart of a process for arranging second imagesaccording to aspects of the disclosure. In operation 1001, the frequencywith which each of the filters (or effects corresponding to the filters)of the plurality of second images is selected is determined. Next, inoperation 1003, an order in which the second images from the pluralityare to be arranged is determined based on the determined selectionfrequencies. Afterwards, in operation 1005, the second images arearranged in the determined order. Although in this example, the order ofsecond images is determined based on the second images' respectivefilters (or effects corresponding to the filters), in other example thelocation on the touch screen 128 where a given image is displayed basedon the frequency with which that second image's respective filter (oreffect) is selected.

FIG. 11 is a flowchart of another process for arranging second imagesaccording to aspects of the disclosure. In operation 1101, one or moresecond images that are associated with similar effects (or filters) areidentified. Next, in operation 1103, the identified second images arearranged in a group. In some implementations, when the second images arearranged in a group, they may be displayed next to each other.

FIG. 12 is a flowchart of a process for displaying second imagesaccording to aspects of disclosure. Referring to FIG. 12, in operation1201, the processor 112 (e.g., the display module 270) may activate atleast one second image that is associated with an effect which is notsynthesizable together with the effect of a second image that has beende-selected (e.g., an image whose selection has been removed byunchecking a checkbox associated with the image). Next, in operation1203, the processor 112 may deactivate at least one second image thathas not been selected based on the deactivated second image beingassociated with an effect which is not synthesizable together with theeffect of a second image that has been selected. In someimplementations, two effects may be considered mutually-synthesizablewhen the filters that implement the effects may be applied on an imageone after another. As a trivial example, an effect that changes all redcolors in an image into blue may not be applicable after an effect thatinvolves transforming the image to a grayscale color space. In someimplementations, when a second image is deactivated, that second imagemay not be selected by the user.

FIGS. 13 to 21 illustrate screens of a camera application according tovarious aspects of the disclosure. In these examples, the cameraapplication may use a camera API provided by an electronic device (e.g.,smart phone) operating system (OS). The camera API may provide variousfilters for modifying images captured by the camera. The cameraapplication, may load those filters on a need basis.

FIG. 13 illustrates an original screen 1300 of the camera application.According to this example, the screen 1300 may display image data thatis being captured by the image sensor 122 in image pane 1310. In someimplementations, the content of the image pane 1310 may be refreshedperiodically, as new images are being captured the image sensor 122. Thescreen 1300 may display an effect menu button 1302. If the effect menubutton 1302 is pressed, the screen 1300 may show icons corresponding toeffects applicable to the image data. In addition, the screen 1300 maydisplay a camera button 1303. If the camera button 1303 is pressed, thescreen 1300 may obtain a snap image. The snap image refers to ahigh-definition image which is obtainable when the camera button 1303 ispressed. When the snap image is obtained, the image pane 1310 may stopbeing refreshed in real-time and the snap-image may be displayed on thescreen permanently, until the user desires to resume the display inreal-time of data captured by the image sensor 122. When the userpresses the effect menu button 1302, the screen 1300 is switched to ascreen 1400 of FIG. 14.

Referring to FIG. 14, if the user presses the effect menu button 1302,the effect menu button 1302 is moved up, and a plurality of icons 1401,1402, 1403, 1404, 1405, and 1406 may appear arranged horizontally at alower side of the screen 1400. The icons 1401, 1402, 1403, 1404, 1405,and 1406 may be arranged horizontally at an upper side of the screen1400 or may be arranged vertically at a left or right side of the screen1400. If the user presses the effect menu button 1302 once again, theeffect menu button 1302 may return to the original position of thescreen 1300 of FIG. 13, and the icons 1401, 1402, 1403, 1404, 1405, and1406 may disappear. If the user touches a region displaying image dataobtained from the image sensor 122, the screen 1400 may return to thescreen 1300 of FIG. 13. In the absence of a selection of the icons 1401,1402, 1403, 1404, 1405, and 1406 within a predetermined time period, thescreen 1400 may return to the screen 1300 of FIG. 13.

The first icon 1401 may display image data obtained by applying noeffect to the image data obtained from the image sensor 122. The othericons 1402, 1403, 1404, 1405, and 1406 may display images that areobtained by processing image data obtained from the image sensor 122with various filters. The icons 1402, 1403, 1404, 1405, and 1406 maydisplay image data obtained by applying effects to the image dataobtained from the image sensor 122, respectively. For example, thesecond icon 1402 may display image data obtained by applying ‘RED’effect to the image data obtained from the image sensor 122. As anotherexample, the third icon 1403 may display image data obtained by applying‘GREEN’ effect to the image data obtained from the image sensor 122.

The first icon 1401 may display a label containing the word ‘ORIGINAL’1411 indicating that the image of this icon is an unfiltered version ofthe image data that is obtained from the image sensor 122. The othericons 1402, 1403, 1404, 1405, and 1406 may display labels identifyingthe button's respective effects (or filters) 1412, 1413, 1414, 1415, and1416 indicating effects, respectively. As noted above, the respectivefilter of a given icon is a filter which may be used to generate theimage of that icon.

The icons 1401-1406 are scrollable. If the user performs a flick gesture(1420) on the icons 1401-1406, those icons may be scrolled to the leftof the screen 1400, and replaced with new icons 1501, 1502, 1503, 1504,1505, and 1506 as seen on a screen 1500 of FIG. 15. If the user selectsthe second icon 1502 in the screen 1500 of FIG. 15, the image dataobtained from the image sensor 122 may be processed by using the filtercorresponding to the second icon 1502 to produce resultant image dataand the resultant image data may be displayed in the image pane 1310. Asillustrated in FIG. 15, in this example, the second button 1502 isassociated with a sketch effect filter.

The second icon 1502 may be selected by any suitable user gesture. Auser's gesture of selecting an icon may be a touch, hovering, a singletouch, a double tap, a drag and drop, flicking 1523, and the like. Forinstance, the user may select the second icon if the user touches andflicks (1523) the second icon 1502 towards point in the image pane 1310,

Referring to FIG. 16, a selected icon 1502 may display a check box 1622indicating to indicate that it has been selected. If the user touchesthe check box 1622, the selection of the icon 1502 may be released. Ifthe user touches and flicks (1623) the region displaying the image dataobtained from the image sensor 122 to the selected icon 1502, theselection of the icon 1502 may be released. If the selection of the icon1502 is released, the check box 1622 may disappear. If the selection ofthe icon 1502 is released, the sketch effect applied to the imagedisplayed in the image pane 1310 is removed. If the selection of theicon 1502 is released, the image pane 1310 may display image dataobtained from the image sensor 122 without modifying that data firstwith one of the filters corresponding to icons 1502-1506. Additionallyor alternatively, in some implementations, if the user touches theaforementioned first icon 1401, the selection of the icon 1502 may alsobe released.

Referring to FIG. 17, if a plurality of different icons are selected, ascreen 1700 may display in the image page 1310 image data obtained byapplying multiple effects to the image data obtained from the imagesensor 122. For instance, if the second icon 1502 and the fifth icon1505 are selected, the image pane 1310 may display image data obtainedby applying a sketch effect and a monochromatic effect to the image dataobtained from the image sensor 122.

Referring to FIG. 18, if one icon is selected many times, a screen 1800may display image data obtained by applying, to image data obtained fromthe image sensor 122, an effect corresponding to the selected icon asmany times as the icon is selected. For instance, if the second icon1502 is selected once and the fifth icon 1505 is selected three times(e.g., by tapping three times on the icon), the image pane 1310 maydisplay resultant image data obtained by applying a sketch effect onceand applying a monochromatic effect three times to image data obtainedfrom the image sensor 122. The selected second icon 1502 and fifth icon1505 may display numerals ‘1’ (1832) and ‘3’ (1835) which indicate thenumber of times the second icon 1502 and the fifth icon 1505 have beenselected, respectively. The once selected second icon 1502 may displayonly a check box 1822 and/or the numeral ‘1’ (1832). If the user touchesa check box 1825 of the fifth icon 1505 once, the screen 1800 may changethe numeral ‘3’ displayed in the fifth icon 1505 into ‘2’, therebycausing the image pane 1310 to display resultant image data that isobtained by applying the filter corresponding to icon 1502 (sketcheffect filter) once and by applying the filter corresponding to the icon1505 (monochromatic filter) twice. In that regard, each of the icons1501-1506 may include user interface components for increasing and/ordecreasing the number of times a filter corresponding to that icon is tobe applied.

Referring to FIG. 19, a screen 1900 is shown in which the number oftimes icons have been selected is indicated by dashes 1932 and 1935superimposed on the icons' images, rather than numerals.

Referring to FIG. 20, an example of a screen 2000 is shown. In thisexample, icons whose corresponding effects are incompatible with theeffects of selected icons are disabled. As illustrated, in this example,icon 1502 is selected and icons 1504 and 1506 are disabled while icons1503 and 1505 are permitted to remain active. As noted above, in someimplementations, two effects (or filters corresponding to the effects)are considered compatible when they are mutually-synthesizable.Additionally or alternatively, two effects (or filters corresponding tothe effects) are considered incompatible when they are notmutually-synthesizable.

Referring to FIG. 21, an example of a screen 2100 is shown in which if asnap image is acquired. When a snap image is acquired, the imagesdisplayed in image pane 1310 and icons 1401, 2102, and 1403-1406 stopbeing refreshed periodically and are transitioned into a state in whichthey display only the acquired (e.g., snapped) image. As illustrated,the icons may display the acquired image as modified by applyingmultiple effects. For instance, if the camera button 1303 is pressed inthe screen 1800 of FIG. 18, the screen 2100 may obtain a snap image, andmay apply the group of effects (i.e., sketch effect level 1, andmonochromatic effect level 3) corresponding to selected icons to thesnap image.

In some implementations, the content of one of the icons may vary independence on what other icons have been selected. For example, thescreen 2100 may display an icon 2102 whose content is based on theidentified group of effects discussed with respect to FIG. 18. Morespecifically, in some instances, the icon 2102 may display image dataobtained by applying the grouped effects to the image data obtained fromthe image sensor 122. Additionally or alternatively, in someimplementations, the icon 2102 may display information 2132 indicatingthe name of two or more of the effects that are applied to image(s)obtained from the image sensor 122 and/or the number of times each ofthe effects is applied.

Additionally or alternatively, in some instances, the locations in thescreen 2100 where an icon is displayed may be changeable in response touser input. For instance, if the user touches and drags the second icon2102 between the fourth icon 1404 and the fifth icon 1405 and thenreleases the drag, the second icon 2102 may be moved between the fourthicon 1404 and the fifth icon 1405. Additionally or alternatively, theuser may move an icon further towards the center of the image pane 1310in the same manner. Additionally or alternatively, the position of atleast one of the icons 1401, 2102, and 1403-1406 in the screen 2100 maybe determined automatically. For instance, an icon of high selectionfrequency may be arranged in a location favorable for a user's access.In some implementations, icons whose respective filters tend to beselected more frequently than the filters of other displayed icons maybe displayed in an upper portion of the screen (while the other iconsare displayed at a lower portion of the screen). Furthermore, in someimplementations, an icon may be rendered unscrollable based on thefrequency with which the filter of the icon is selected exceeding apredetermined threshold, thus resulting in the user being able to scrollother icons, as discussed with respect to FIG. 14, but not the iconwhose filter's selection frequency has exceeded the threshold.Furthermore, in some implementations, icons of similar effects may bearranged in a group. For instance, in FIG. 14, the icon 1402 of ‘RED’effect, the icon 1403 of ‘GREEN’ effect, and the icon 1404 of ‘BLUE’effect are all for adjusting a color sense and thus, they may bearranged in a group (e.g., by being placed next to each other). Acriterion for determining a similarity of effects may be brightness,chroma and the like.

Additionally or alternatively, in some implementations, the cameraapplication may provide a filter addition function. Through the filteraddition function, a user may obtain a filter from an external, such asa remote server. The filter may be obtained via a communications networkconnection, e-mail, messenger, a detachable external memory, and or anyother suitable method. The filter obtained from the external may bestored in a designated directory, and may be commonly used byapplications. The camera application may add an icon corresponding tothe filter obtained from the external. This icon may display image dataobtained by applying the effect of the filter to image data obtainedfrom the image sensor 122.

Additionally or alternatively, in some implementations, a user maydownload a filter package file of an application type from a network,and may install the filter package file. If the filter package file isinstalled, a filter included in the filter package file may be stored ina designated directory such that applications may make common use of thefilter. This filter may be automatically added to a filter list of thecamera application, and an icon corresponding to the added filter may beadditionally displayed through the aforementioned scheme.

FIGS. 4-11 are provided as examples only. At least some of theoperations discussed with respect to these Figures may be executed inparallel, in different order, or altogether omitted. Any of theoperations of discussed with respect to FIGS. 4-11 may be executed byprocessing circuit, such as a processor 112. Furthermore, in someimplementations, different ones of the operations discussed with respectto FIGS. 4-11 may be executed by different processing circuitry modules.It will be readily appreciated, that the processes discussed withrespect to FIGS. 4-11 may be implemented in hardware, in software (e.g.,as processor-executable instructions executed by a processor), or as acombination of hardware and software.) It will be further understoodthat the terms “effect” and “filter” are used interchangeably throughoutthe disclosure.

The above-described embodiments of the present disclosure may beimplemented in hardware, firmware or via the execution of software orcomputer code that may be stored in a recording medium such as a CD ROM,a Digital Versatile Disc (DVD), a magnetic tape, a RAM, a floppy disk, ahard disk, or a magneto-optical disk or computer code downloaded over anetwork originally stored on a remote recording medium or anon-transitory machine readable medium and to be stored on a localrecording medium, so that the methods described herein may be renderedvia such software that is stored on the recording medium using a generalpurpose computer, or a special processor or in programmable or dedicatedhardware, such as an ASIC or FPGA. As would be understood in the art,the computer, the processor, microprocessor controller or theprogrammable hardware include memory components, e.g., RAM, ROM, Flash,etc. that may store or receive software or computer code that whenaccessed and executed by the computer, processor or hardware implementthe processing methods described herein. In addition, it would berecognized that when a general purpose computer accesses code forimplementing the processing shown herein, the execution of the codetransforms the general purpose computer into a special purpose computerfor executing the processing shown herein. Any of the functions andsteps provided in the Figures may be implemented in hardware, softwareor a combination of both and may be performed in whole or in part withinthe programmed instructions of a computer. No claim element herein is tobe construed under the provisions of 35 U.S.C. 112, sixth paragraph,unless the element is expressly recited using the phrase “means for”.

While the disclosure has been shown and described with reference tocertain preferred aspects thereof, it will be understood by thoseskilled in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the disclosure asdefined by the appended claims.

What is claimed is:
 1. A method for displaying images, the methodcomprising: displaying, by an electronic device on a touch screen, afirst image based on image data obtained from an image sensor, and aneffect menu button, wherein the first image is displayed as a largesized preview image, occupying a majority area of the touch screen,refreshed based on image data obtained in real-time from the imagesensor; detecting a touch input for selecting the effect menu button onthe touch screen; responsive to detecting the touch input for selectingthe effect menu button, displaying, by the electronic device, aplurality of second images and the first image, wherein each of thesecond images is displayed as a thumbnail sized preview image refreshedby applying a respective image effect to image data obtained inreal-time from the image sensor; detecting a touch input for selectingat least one of the second images on the touch screen; responsive todetecting the touch input for selecting the at least one of the secondimages, applying a respective image effect of the at least one selectedsecond image to image data obtained from the image sensor to generate athird image; displaying the third image together with the second images,wherein the third image is displayed in place of the first image as alarge sized preview image refreshed based on image data obtained inreal-time from the image sensor; receiving a touch input for capturing astill image; and generating, in response to the touch input, a stillimage by applying the respective image effect of the third image toimage data obtained from the image sensor after receiving the touchinput, wherein the second images are refreshed as preview imagethumbnails together with the third image as a preview image at leastuntil the touch input for capturing the still image is received.
 2. Themethod of claim 1, wherein the generating of the still image generatesthe still image with a resolution higher than that of the third image.3. The method of claim 1, further comprising: determining whether thetouch input for selecting at least one of the second images is receivedwithin a specified period of time; and terminating the displaying of theplurality of second images based on a determination that the touch inputis not received within the specified period of time.
 4. The method ofclaim 1, further comprising: detecting a touch input with respect to theeffect menu button while the plurality of second images are displayed;and in response to the touch input, terminating the displaying of theplurality of second images.
 5. The method of claim 1, furthercomprising: displaying a second image to which no filter effect isapplied at a predetermined position among the plurality of secondimages.
 6. The method of claim 1, further comprising: displaying anindication indicating that no filter effect is applied to the secondimage to which no filter effect is applied.
 7. The method of claim 1,further comprising: displaying, in relation with each of the pluralityof second images, an indication indicative of the respective imageeffect in a specified language.
 8. The method of claim 1, furthercomprising: determining if the same second image is selected multipletimes; when it is determined that the second image is selected multipletimes, displaying an indication of a number of times one of the secondimages is selected and generating the third image by applying, to afirst image, the effect corresponding to the second image as many timesas the second image is selected; and when it is determined that thesecond image is selected a single time, generating the third image byapplying once, to the first image, the image effect corresponding to theselected one of the second images.
 9. The method of claim 1, furthercomprising: determining whether multiple different second images areselected; and when multiple different second images are selected,generating the third image by applying the effects corresponding to therespective selected multiple different second images to the first image.10. The method of claim 1, further comprising deactivating for selectionat least one 10 second image that has not been selected based on thatsecond image being associated with an effect which is not compatiblewith the effect corresponding to another second image that has beenselected.
 11. The method of claim 1, further comprising activating forselection at least one 15 second image that is associated with an effectwhich is not compatible with the effect of another second image that hasbeen de-selected.
 12. The method of claim 1, wherein the second imagesare displayed in an order that is determined based on a characteristicof the second images' respective image effects, wherein thecharacteristic is selected from a group including color, brightness, andchroma.
 13. The method of claim 1, wherein the second images aredisplayed at a location determined based on a frequency with which thatsecond images' respective image effects have been selected in a pastperiod of time.
 14. The method of claim 1, further comprisingdeactivating one of the second images based on a selection of anotherone of the second images, wherein the deactivation renders thedeactivated second image incapable of being selected.
 15. An electronicdevice comprising: a memory storing instructions a touch screen; animage sensor; and one or more processor configured to execute the storedinstructions to: display, on the touch screen, a first image based onimage data obtained from an image sensor, and an effect menu button,wherein the first image is displayed as a large sized preview image,occupying a majority area of the touch screen, refreshed based on imagedata obtained in real-time from the image sensor; detect a touch inputfor selecting the effect menu button on the touch screen; responsive todetecting the touch input for selecting the effect menu button, displaya plurality of second images and the first image, wherein each of thesecond images is displayed as a thumbnail sized preview image refreshedby applying a respective image effect to image data obtained inreal-time from the image sensor; detect a touch input for selecting atleast one of the second images on the touch screen; responsive todetecting the touch input for selecting the at least one of the secondimages, apply a respective image effect of the at least one selectedsecond image to image data obtained from the image sensor to generate athird image; display the third image together with the second images,wherein the third image is displayed in place of the first image as alarge sized preview image refreshed based on image data obtained inreal-time from the image sensor; receive a touch input for capturing astill image; and generate, in response to the touch input, a still imageby applying the respective image effect of the third image to image dataobtained from the image sensor after receiving the touch input, whereinthe second images are refreshed as preview image thumbnails togetherwith the third image as a preview image at least until the touch inputfor capturing the still image is received.
 16. The electronic device ofclaim 15, wherein the one or more processor is further configured togenerate the still image with a resolution higher than that of the thirdimage.
 17. The electronic device of claim 15, wherein the one or moreprocessor is further configured to: determine whether the touch inputfor selecting at least one of the second images is received within aspecified period of time; and terminate the displaying of the pluralityof second images based on a determination that the touch input is notreceived within the specified period of time.
 18. The electronic deviceof claim 15, wherein the one or more processor is further configured to:detect a touch input with respect to the effect menu button while theplurality of second images are displayed; and in response to the touchinput, terminate the displaying of the plurality of second images. 19.The electronic device of claim 15, wherein the one or more processor isfurther configured to display a second image to which no filter effectis applied at a predetermined position among the plurality of secondimages.
 20. The electronic device of claim 15, wherein the one or moreprocessor is further configured to display an indication indicating thatno filter effect is applied to the second image to which no filtereffect is applied.
 21. The electronic device of claim 15, wherein theone or more processor is further configured to display, in relation witheach of the plurality of second images, an indication indicative of therespective image effect in a specified language.
 22. The electronicdevice of claim 15, wherein the one or more processor is furtherconfigured to: determine if the same second image is selected multipletimes; when it is determined that the second image is selected multipletimes, display an indication of a number of times one of the secondimages is selected and generating the third image by applying, to afirst image, the effect corresponding to the second image as many timesas the second image is selected; and when it is determined that thesecond image is selected a single time, generate the third image byapplying once, to the first image, the image effect corresponding to theselected one of the second images.
 23. The electronic device of claim15, wherein the one or more processor is further configured to:determine whether multiple different second images are selected; andwhen multiple different second images are selected, generate the thirdimage by applying the effects corresponding to the respective selectedmultiple different second images to the first image.
 24. The electronicdevice of claim 15, wherein the one or more processor is furtherconfigured to deactivate for selection at least one second image thathas not been selected based on that second image being associated withan effect which is not compatible with the effect corresponding toanother second image that has been selected.
 25. The electronic deviceof claim 15, wherein the one or more processor is further configured toactivate for selection at least one second image that is associated withan effect which is not compatible with the effect of another secondimage that has been de-selected.
 26. The electronic device of claim 15,wherein the second images are displayed in an order that is determinedbased on a characteristic of the second images' respective imageeffects, wherein the characteristic is selected from a group includingcolor, brightness, and chroma.
 27. The electronic device of claim 15,wherein the second images are displayed at a location determined basedon a frequency with which that second images' respective image effectshave been selected in a past period of time.
 28. The electronic deviceof claim 15, wherein the one or more processor is further configured todeactivate one of the second images based on a selection of another oneof the second images, wherein the deactivation renders the deactivatedsecond image incapable of being selected.